Contrary to common belief, the most fundamental challenge to the emergent knowledge based economy is not the creation of new knowledge but rather is the problem of enabling individuals to access and manipulate knowledge that is new to the individual. In order for this to occur, individuals must learn the new knowledge and understand how to manipulate it. In today's environment, this should be done by all individuals, from elementary students to secondary students, to post-secondary students and adults in the workplace.
The conventional learning process can be understood in the context of an instructor and a student. The instructor teaches material that is to be learnt by the student and then assesses how well the student understands the material. This traditional teaching/evaluation process comprises three phases:
(a) Input Phase: In this phase, there is knowledge transfer from the instructor to the student. The instructor uses any perceptual means necessary to transfer the knowledge and meaning of the material being taught to the student. In this stage, the student is a listener, reader or watcher. This is the input stage from the student's perspective.
(b) Processing phase: In this phase, the student has access to the material and is trying to understand what has been taught. This is a stage in which the student is thinking to evaluate the input and construct and restructure his knowledge. During this phase, the student may also pose questions to the instructor, discuss issues and exchange opinions with his peers, and seek advice and clarification from other individuals as well as from other sources.
(c) Testing phase: In this phase, the instructor wants to determine what and how much of the material has been learnt and thoroughly understood by the student. In this phase, the student is generating information and the instructor is processing this information to determine whether, and to what extent, the student understands the material.
Unfortunately, there are shortcomings with this traditional learning process. The initial phase is problematic because the student may not understand the material being taught in the manner in which the instructor presents the material. For any one student, it may be beneficial to present the material in an alternative fashion or another context to allow the student to see the material from a different perspective and thus attain a deeper level of comprehension.
Another shortcoming is the testing process in which the student is usually asked to reproduce or recognize some part of the material that has been taught. Unfortunately, this testing methodology encourages the student to memorize the material, sometimes without understanding it, and simply reproduce the material or apply an algorithm or a well-rehearsed rule without necessarily understanding the underlying concepts. Furthermore, instructors have also been known to teach to the test such that the student will be trained on problems which will be on the test. This allows the student to perform well on the test and gives the impression that the instructor is a good teacher which may not necessarily be the case. In addition, the testing process (e.g. asking multiple choice questions, true/false questions, etc.) often involves asking the student questions about details such that the answers provided by the student to these questions often do not reveal the depth of comprehension that the student has acquired for the material that has been taught.
Both of these problems are exasperated in the post-secondary environment because there may be as many as 1000 to 2000 students enrolled simultaneously in the same course. This large number of students prevents adequate instructor/student interaction from occurring. Typically only a few students ask the instructor questions or see a teaching assistant for help. Thus, the input phase of learning is limited.
Furthermore, the large number of students taking a given course places limitations on the testing phase. Typically, in order to test such a large number of students, the instructor uses a multiple choice test for which the student may cram, memorize material verbatim and may subsequently receive a high mark. Thus, these tests provide little value to the instructor in determining how knowledgeable the student is. Alternatively, essay questions may be used in the testing process but these are difficult to mark since, in a large enrolment course with several teaching assistants, different teaching assistants mark the answers to the essay questions and each may have an inherently unique subjective marking criteria. This prevents the uniform marking of answers to essay questions.
These problems in the assessment of deep comprehension of newly learned concepts have been widely recognized. In response, there have been attempts by instructors to develop alternative methods for probing students to determine their true level of deep understanding. These methods include: Concept Mapping, Prediction-Observation-Explanation, Interviews about Instances and Events, Interviews about Concepts, Drawings, Fortune Lines, Relational Diagrams, Word Association and Question Production. However, none of these educational methods have been shown to assess deep understanding with high construct validity and reliability. A high reliability test refers to a test which is administered several times and produces similar results each time. A test with construct validity refers to a test which correctly measures or tests for the desired construct.
One attempt at assessing deep understanding was made by Royer and his associates (Royer, J. M. (1990), “The Sentence Verification Technique: A new direction in the assessment of reading comprehension.” in S. M. Legg and J. Algina (Eds.), Cognitive assessment of language and math outcomes, pp. 144-191; Royer, J. M. (1997), “Cognitive perspective on the assessment, diagnosis, and remediation of reading skills.” in G. D. Phye (Ed.), Handbook of academic learning, pp. 199-234; Royer, J. M., & Carlo, M. S. (1993), “Assessing language comprehension skills in cross-cultural settings.” in J. Altarriba (Ed.), Cognition and culture: A cross-cultural approach to psychology, pp. 157-175). Royer teaches a new method, the Sentence Verification Test (SVT), for assessing the listening and reading comprehension of students for paragraphs and sentences. In the reading version of the SVT, after reading a paragraph consisting of 12 sentences, the student is presented with several test sentences and is asked to identify each of the test sentences as either being an old sentence (i.e. the test sentence is similar to one of the sentences in the paragraph that was just read) or a new sentence. The old sentences may either be the original sentences in the paragraph or paraphrases of the sentences that appeared in the paragraph. The paraphrased sentences are created by changing as many words as possible in the original sentence without altering the meaning of the original sentence. The new sentences have a different meaning compared to the sentences in the paragraph. The different meaning may be due to changing one or two words in the original sentence in the paragraph. The new sentences may also be distracters which are sentences that are syntactically and thematically similar to the original sentences in the paragraph but do not share their meaning. Royer also teaches another version of the SVT called the Meaning Identification Technique (MIT) in which only paraphrased sentences or sentences with different meanings, compared to the sentences in the original paragraph, are presented to the student following the reading of the paragraph. SVT and MIT have been shown to be good measures of listening and reading comprehension, text readability, reading skill, and working memory. However, by providing the student with a target statement, i.e. the original paragraph, these testing methods are prone to being a memory-for-text and not a test for deep comprehension of content.
While there has been much effort and emphasis on developing new learning/testing methods for students in the elementary, secondary and post-secondary environments, the adult education and corporate training environments have currently not been widely recognized as a scholarly discipline worthy of a similar effort. However, the need for adult learning is gaining recognition as an important social goal and is attaining high priority. One of the driving factors for adult learning is the fast changing workplace which requires innovative education and training programs to aid corporations in maintaining competitiveness by using efficient and cost-effective training methods to educate employees. These methods are also beneficial in helping employees ease transition-induced stress by facilitating the learning process and by optimizing learning outcomes. However, in developing adult training methods, one must be careful to avoid the shortcomings of the traditional learning process that were just discussed.
Accordingly, there still remains a need for an improved testing methodology that can assess the deep understanding of a student after the student has been taught material with new conceptual content in a given subject area. It would also be beneficial if this test methodology could be used as a tool that enhances learning. In addition, it would be beneficial if the test methodology could be extended to adult learning and to corporate environments. It would be further beneficial if the test methodology could use any learning delivery mode, i.e. classroom-based instruction, as well as Internet and computer-based instruction.